restructure actions to reusable build flow with parameters #1779

deitch · 2025-10-03T07:51:33Z

Here is a first cut of #1778, as discussed with @cmuellner. I will describe what it does here. If you think that warrants a README inside the .github/workflows/ directory, happy to oblige.

Before, we had multiple build runs, each redoing the same steps. They occurred in:

main build (in build.yml): (full matrix of variants)
sim (in build.yml): just one variant, but with the --with-sim arg
multilib (in build.yml): two variants
nightly (in nightly-release.yml): full matrix of variants

In addition, one created releases, some created artifacts, some did not

This restructures it as follows.

First, there is a single reusable workflow, defined in build-reusable.yml. It can get called by any other workflow. It has multiple args:

the artifact name
all of the matrix elements (os, mode, target, compiler), each of which has defaults identical to the current full usage
sim: to enable --with-sim

The other workflows - regular build, multilib, test-sim, nightly - all call it with the correct args.

In addition, I did a few more "cleanup" changes.

Replaced the larger nightly release creation cycle with the updated softprops/create-gh-release action, which includes everything needed. The original GitHub one is archived anyways and points at several replacements, including create-gh-release. This just means much shorter code.
Replaced the logic for activity check ("is it updated in last 24 hours?") from one that used the GitHub API to one that just clones the repo and checks the commits locally. Functionally, it is the same, but avoids dealing with the API and all of the jq parsing.
updated usage of steps outputs rather than environment, as recommended by GitHub

The reusable build and parameters make it much easier to test variants. You even could dispatch a build manually using a different set of parameters for the matrix, e.g. build on a different OS based or different compiler (I have no idea which one, but if you can think of one).

Looking forward to your feedback.

deitch · 2025-10-03T07:52:20Z

Just fair warning: GitHub Actions workflows changes tend to be really good when complete, and the biggest pain to get right, because you often cannot tell if it worked fully until post-merge. I am hoping it won't be too difficult here.

deitch · 2025-10-03T08:02:55Z

BTW, while we are doing this, I see that multilib is disabled.

If I already am doing this, do you want a parameter that disables any set of workflows? We could do something simple like (pseudocode): if DISABLE_BUILD != "" then do not run the build job and if DISABLE_MULTILIB != "" then do not run the multilib job, etc for each one. Then you can enable/disable jobs entirely from the GitHub actions variables, without having to change code. It also makes it much easier for people working with forks until they upstream, since they do not need to modify them not to run in their own environment, just set variables in settings.

I like the idea, but am asking before adding it into the PR.

Copilot

Pull Request Overview

This PR restructures the GitHub Actions workflows to use a single reusable build workflow with parameters, consolidating previously duplicated build logic across multiple workflows.

Introduces a new reusable workflow (build-reusable.yaml) that accepts parameters for OS, mode, target, compiler, and simulator options
Replaces individual build configurations in build.yaml and nightly-release.yaml with calls to the reusable workflow
Simplifies the nightly release process by using a modern GitHub release action and local git operations instead of API calls

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File	Description
`.github/workflows/build-reusable.yaml`	New reusable workflow that consolidates build logic with configurable matrix parameters
`.github/workflows/build.yaml`	Simplified to call the reusable workflow with appropriate parameters for different build types
`.github/workflows/nightly-release.yaml`	Updated activity check logic and release creation using modern actions

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

.github/workflows/nightly-release.yaml

.github/workflows/build-reusable.yaml

deitch · 2025-10-03T09:33:45Z

Good copilot catches, fixed.

deitch · 2025-10-03T14:07:02Z

I saw the build failed. I'll dig into it early next week.

TommyMurphyTM1234 · 2025-10-03T14:21:56Z

Might this PR also fortuitously fix this issue or does that need separate attention?

CI nightly build email doesn't list all assets #1656

deitch · 2025-10-03T14:33:47Z

Not sure. Once I get this working correctly, I don't mind looking at that.

What sends out the emails?

TommyMurphyTM1234 · 2025-10-03T15:20:14Z

Not sure. Once I get this working correctly, I don't mind looking at that.

Thanks.

What sends out the emails?

I thought that it was some standard part of the GitHub action? I looked into this issue a little a while back but can't remember the details unfortunately.

deitch · 2025-10-05T06:51:40Z

OK, figured out that particular failure, fixed and pushed. It needs approval to rerun the workflow.

It was nice to see that the basic build jobs all ran correctly, just test-sim failed, and hopefully I got that. build-multilib was disabled anyways. Once I am sure those are all building correctly, I will make sure the artifacts are named and uploaded correctly, after someone approves another workflow run.

deitch · 2025-10-05T06:52:10Z

Any thoughts on the DISABLE_JOB_* env var approach?

deitch · 2025-10-05T07:02:08Z

Might this PR also fortuitously fix this issue or does that need separate attention?

I don't see anything in the CI jobs that sends an email. There only are two CI workflows:

build.yaml, which builds on each PR or merge-to-master (or manual kickoff)
nightly-release.yaml which does the nightly release at midnight

Looking at the latter (the only real candidate), it is set to

run at midnight, during which it checks if any new commits to master in last 24 hours
running the build
creating the release
uploading assets.

There is nothing there about sending out an email, so it is not something generic.

It is possible that you as maintainers, as well as anyone else fully following the repository, automatically gets emails for any new release. I know I get those for other projects/repos that I maintain.

Given how this is structured - first, create the release (which should include source automatically), then, in a separate job in the workflow, upload the assets - that the email is getting triggered by the first before the second is done, hence including just the source code assets.

If so, switching to the actions-gh-release, which includes all of the assets, might successfully fix that. It depends on how the action does it under the covers.

That does make sense, but cannot be proven until it is merged in. I would recommend waiting until this is merged in, and seeing what happens with first midnight post-merge. If it fixes it, great, we know that was the issue; if not, can dig deeper or elsewhere.

Copilot

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

.github/workflows/build-reusable.yaml

kito-cheng

The last comment from copilot seems not

.github/workflows/build-reusable.yaml

deitch · 2025-10-07T18:23:49Z

Fixes the raised issues. Could use another workflow run.

deitch · 2025-10-08T07:18:46Z

This is an improvement. All of the runs are running. It is failing solely because of conflict naming in uploaded artifacts. I should be able to figure those out.

Copilot

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

.github/workflows/nightly-release.yaml

.github/workflows/build-reusable.yaml

Signed-off-by: Avi Deitcher <[email protected]>

deitch · 2025-10-08T08:29:56Z

Fixed the copilot nits. Also I believe fixed the artifact naming issue. Ready for another workflow run.

Copilot

Pull Request Overview

Copilot reviewed 3 out of 3 changed files in this pull request and generated 2 comments.

_{Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.}

Copilot · 2025-10-08T12:53:18Z

.github/workflows/nightly-release.yaml

      - name: Download Built Artifacts
        uses: actions/download-artifact@v4
        with:
          path: ${{ env.ARTIFACTS_DIR }}


The ARTIFACTS_DIR environment variable is not defined at this point. It should be moved from the create-release job to the top-level env section or defined before this step.

.github/workflows/build-reusable.yaml

kito-cheng

Just one nit from me, otherwise LGTM

kito-cheng · 2025-10-08T12:58:48Z

.github/workflows/nightly-release.yaml

        shell: bash

-
+  


Seems adding trailing spaces

Suggested change

I will fix that, but I want to let the build run to go through first, make sure there are no other issues.

deitch · 2025-10-08T21:30:42Z

This looks pretty good, all of the builds that were supposed to run, did so.

The remaining thing is the nightly. It only kicks off midnight, and only if there were changes in the last 24 hours (which there will be after the merge, by definition). As soon as you merge, I will check the next morning, and if there are any issues, I will open a PR to fix them.

Please don't merge on a Friday, as I won't see it on Saturday morning. I also will be out of touch this coming Tuesday, so please don't merge this coming Monday.

deitch · 2025-10-09T16:26:06Z

@kito-cheng @cmuellner anything else I need to do here to make it good?

deitch · 2025-10-14T18:38:36Z

OK I can check post-merge over the coming days. When can we move in, or what is missing?

kito-cheng · 2025-10-16T13:45:29Z

hi @deitch:

Thanks for your PR! nothing block this just because I am working on other stuffs in past few day and miss that in my gmail inbox...and I notice this because #1786 :P

deitch · 2025-10-16T13:47:22Z

I notice this because #1786

My 🤦‍♂️ mistake? 😆

Thanks. I am putting in my calendar to check this out shortly after midnight, see if anything in nightly breaks, so I can open fixes if needed.

deitch · 2025-10-17T08:20:15Z

The nightly release job failed, but due to secondary directory permissions. I will figure it out and add a PR shortly.

The part I really was concerned about - the update to checking activity in the last 24 hours, ensuring it found and downloaded all of the artifacts - all worked correctly, which is nice.

deitch · 2025-10-17T08:29:51Z

Ah that explains it. We tried to download to /mnt/artifacts, but you cannot create that unless you are root. No idea why we do it there. We should just as easily do it somewhere else. I will open a PR shortly.

deitch · 2025-10-17T09:14:50Z

OK got it. Opening a new PR.

deitch · 2025-10-17T09:20:19Z

See #1787

kito-cheng requested a review from Copilot October 3, 2025 09:26

Copilot AI reviewed Oct 3, 2025

View reviewed changes

.github/workflows/nightly-release.yaml Outdated Show resolved Hide resolved

.github/workflows/nightly-release.yaml Show resolved Hide resolved

.github/workflows/build-reusable.yaml Show resolved Hide resolved

deitch force-pushed the reusable-actions-flows branch from 180bef7 to d4f4f53 Compare October 3, 2025 09:33

deitch force-pushed the reusable-actions-flows branch from d4f4f53 to 84b81ed Compare October 5, 2025 06:50

kito-cheng requested a review from Copilot October 7, 2025 01:47

Copilot AI reviewed Oct 7, 2025

View reviewed changes

.github/workflows/build-reusable.yaml Show resolved Hide resolved

.github/workflows/build-reusable.yaml Outdated Show resolved Hide resolved

kito-cheng reviewed Oct 7, 2025

View reviewed changes

.github/workflows/build-reusable.yaml Show resolved Hide resolved

deitch force-pushed the reusable-actions-flows branch from 84b81ed to ad54d04 Compare October 7, 2025 18:23

kito-cheng requested a review from Copilot October 8, 2025 08:03

Copilot AI reviewed Oct 8, 2025

View reviewed changes

.github/workflows/nightly-release.yaml Outdated Show resolved Hide resolved

.github/workflows/build-reusable.yaml Outdated Show resolved Hide resolved

.github/workflows/build-reusable.yaml Outdated Show resolved Hide resolved

deitch force-pushed the reusable-actions-flows branch from ad54d04 to db25153 Compare October 8, 2025 08:28

restructure actions to reusable build flow with parameters

ff49d85

Signed-off-by: Avi Deitcher <[email protected]>

deitch force-pushed the reusable-actions-flows branch from db25153 to ff49d85 Compare October 8, 2025 08:29

kito-cheng requested a review from Copilot October 8, 2025 12:52

Copilot AI reviewed Oct 8, 2025

View reviewed changes

kito-cheng approved these changes Oct 8, 2025

View reviewed changes

kito-cheng requested a review from cmuellner October 8, 2025 13:01

deitch mentioned this pull request Oct 16, 2025

Reusable workflows #1786

Closed

kito-cheng merged commit 8094d3e into riscv-collab:master Oct 16, 2025
31 checks passed

deitch deleted the reusable-actions-flows branch October 16, 2025 13:47

restructure actions to reusable build flow with parameters #1779

restructure actions to reusable build flow with parameters #1779

Uh oh!

Conversation

deitch commented Oct 3, 2025

Uh oh!

deitch commented Oct 3, 2025

Uh oh!

deitch commented Oct 3, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

deitch commented Oct 3, 2025

Uh oh!

deitch commented Oct 3, 2025

Uh oh!

TommyMurphyTM1234 commented Oct 3, 2025

Uh oh!

deitch commented Oct 3, 2025

Uh oh!

TommyMurphyTM1234 commented Oct 3, 2025

Uh oh!

deitch commented Oct 5, 2025

Uh oh!

deitch commented Oct 5, 2025

Uh oh!

deitch commented Oct 5, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

kito-cheng left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

deitch commented Oct 7, 2025

Uh oh!

deitch commented Oct 8, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

deitch commented Oct 8, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Uh oh!

Copilot AI Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kito-cheng left a comment

Choose a reason for hiding this comment

Uh oh!

kito-cheng Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

deitch Oct 8, 2025

Choose a reason for hiding this comment

Uh oh!

deitch commented Oct 8, 2025

Uh oh!

deitch commented Oct 9, 2025

Uh oh!

deitch commented Oct 14, 2025

Uh oh!